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Just 10 weeks ago, a consortium of researchers from Arizona's universities and research 
institutes released a comprehensive report indicating that a lack of good data has the state 
largely in the dark about the effectiveness of its education policies. 

This was true of policies regarding English-language learners, too, as noted in a chapter I 
co-authored with other researchers. 

We reported that available data "are not suitable to evaluate the effects of specific 
policies for ELL students," and that "no reliable or meaningful conclusions" could follow 
from them. 

Like other authors of the report, we recommended that the state base its policy decisions 
on scientific evidence, separating research as far from politics as possible, and made 
recommendations for achieving this goal. 

Regrettably, concerns raised over the adequacy of available data did not prevent state 
schools Superintendent Tom Horne from releasing a study of the anti-bilingual education 
policy upon which his campaign for office had been based, using the very data 
researchers had warned about. 

Home's study concluded that his department's vigorous ban on Spanish was working 
wonders for students. "There is not a single exception," Horne told The Republic. "It tells 
us that the students in English immersion do substantially better." 

A look at the study reveals that this is a phenomenal exaggeration. 
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Spanish-background students in all-English classrooms in grades affected by the anti- 
bilingual law had anywhere from no advantage to a "two month" advantage over students 
taught bilingually. 

In most cases, immersion students in this group revealed a mere "one month" advantage. 
Due to extreme limitations in the state's education data and flawed analysis in the study, 
even modest boasting would be unjustified. 

Here are some reasons why: 

The study ignored socio-economic differences. Students who attend schools with higher 
poverty levels tend to have lower scores than students who attend wealthier schools. 
Because the study did not consider this factor, we simply do not know whether the 
reported gains are associated with greater resources or with program placement, as 
claimed. 

The study ignored relevant background knowledge. For example, if students in one group 
tended to know more English or have more years of schooling in prior years, they'd end 
up with higher scores regardless of what program they were in. 

The study confused classroom-level and program-level descriptions. For example, in 
bilingual programs, students are taught in bilingual classrooms in early grades and 
transitioned to all-English classrooms after linguistic and academic benchmarks have 
been met. In these circumstances, the highest-scoring bilingual program students would 
be mixed in with the immersion program students, artificially inflating their average 
score. 

That's all bad enough, but things get worse. The study took at face value potentially 
incorrect "program placement" data, which have been found to shift erratically from year 
to year. Thus, the indicator around which the study revolves is itself highly unreliable, 
frequently coded incorrectly by students and teachers who feel confused, pressured and 
fearful about the new law. 

Arizona has become the single most regressive and language-restrictionist state in the 
nation. 

Although its policies are at odds with good program-evaluation research, the Department 
of Education continues to overregulate application of the English-only law, making other 
viable and promising approaches essentially impossible. 

And after four years, the public remains totally ignorant about the consequences of these 
extreme policies for English-language learners in Arizona. 



Jeff MacSwan is an associate professor in the College of Education at Arizona State 
University, and a visiting scholar in the linguistics department at the Massachusetts 
Institute of Technology. 
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